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(54) Tide: MOTION VECTOR FIELD CODING 



(57) Abstract 

A video codec includes a mo- 
tion field coder for minimizing the 
number of motion coefficients of a 
motion vector field. In the coder, a 
first block includes means for form- 
ing a new matrix representation of 
the motion vector field. The new 
coded motion vector field is lin- 
ear. A second main block includes 
means for merging pairs of adjacent 
segments if the combined segment 
area can be predicted using a com- 
mon motion field. Merging infor- 
mation is transmitted to a decoder. 
A third main block includes means 
for removing motion field basis func- 
tions. After each removing step, the 
squared prediction error is calculated 
and removing is continued until the 
chance of the error is not acceptable. 
Final motion coefficients are calcu- 
lated by solving a linear matrix equa- 
tion. As a result, reduced number 
of motion coefficients of each seg- 
ment are obtained. The motion co- 
efficients are transmitted to the de- 
coder. 
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Motion vector field coding 

FIELD OF THE INVENTION 

The present invention generally relates to video compression. More pre- 
5 cisely, the invention relates to a method of coding an estimated motion field 
and for generating motion information in a video sequence. 

BACKGROUND OF THE INVENTION 

The motion compensated prediction is a key element of majority of video 

10 coding schemes. Figure 1 is a schematic diagram of an encoder for compres- 
sion of video sequences using motion compensation. Essential elements in the 
encoder are a motion compensated prediction block 1 , a motion estimator 2 
and a motion field coder 3. The operating principle of the motion compensating 
video coders is to compress the prediction error E n (x,y), which is a difference 

15 between the incoming frame l n (x,y) being coded called the current frame and a 
prediction frame I n (x,y), wherein: 

E n (x,y) = I n (x,y)-i n (x,y) (1) 
The prediction frame 7 n O,;y) is constructed by the motion compensated predic- 
tion block 1 and is built using pixel values of the previous, or some other already 

20 coded frame denoted I„^(x,y), called a reference frame, and the motion vec- 
tors of pixels between the current frame and the reference frame. Motion vec- 
tors are calculated by the motion field estimators 2 and the resulting vector field 
is then coded in some way before applying to the predictor block 1. The predic- 
tion frame is then: 

25 /„ (*, y) = [x + Ax(x, y) 9 y + Ay(x 9 y)} (2) 

The pair of numbers [x+Ax(x,y), y+Ay(x,y)] is called the motion vector of pixel in 
location (x, y) in the current frame, whereas Ax(x,y) and Ay(x,y) are the values of 
horizontal and vertical displacement of this pixel. Set of motion vectors of all 
pixels in the current frame l n (x,y) is called motion vector field. The coded motion 

30 vector field is also transmitted as motion infromation to the decoder. 
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In the decoder, fig 2, pixels of the current frame l n (x,y) are reconstructed 
by finding the pixels' predictions I n (x,y) in the reference frame /„_, (*,>>). The 
motion compensated prediction block 21 generates the prediction frame using 
the received motion information and the reference frame /„_,(*, (in this 
5 picture the reference frame is the same as the current frame). In the prediction 
error decoder 22 decoded prediction error E„(x,y) is then added with the 
prediction frame, the result being the original current frame l n . 

The general object of the motion compensated (MC) prediction is to 
minimize amount of information which needs to be transmitted to the decoder. It 
1 0 should minimize the amount of prediction error measured, e.g., as the energy of 
En(x,y), and minimize the amount of information needed to represent motion 
vector field. 

The document H.Nguen, E.Dubois, "Representation of motion 
information for image coding". Proc. Picture Coding Symposium '90, 

15 Cambridge, Massachusetts, March 26-18, 1990, pages 841-845, gives a 
review of motion field coding techniques.As a rule of the thumb reduction of 
prediction error requires more sophisticated motion field, i.e., more bits must 
be spent on its encoding. Therefore the overall goal of the video encoding is 
to encode as compactly as possible the motion vector field keeping at the 

20 same time the measure of prediction error as low as possible. 

The motion field estimation block 1, fig. 1, calculates motion vectors of 
all the pixels of a given segment which minimize some measure of prediction 
error in this segment, for example square prediction error. Motion field 
estimation techniques differ both in the model of the motion field and in the 

25 algorithm for minimisation of the chosen measure of prediction error. 

Due to very large number of pixels in the frame it is not efficient to 
transmit a separate motion vector for each pixel. Instead, in most of the video 
coding schemes the current frame is divided into larger image segments so 
that all motion vectors of the segment can be described by few parameters. 

30 Image segments can be square blocks, e.g. 16x 16 pixels blocks are used in 
codecs in accordance with international standard ISO/IEC MPEG-1 or ITU-T 
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H.261 , or they can comprise of completely arbitrarily shaped regions obtained 
for instance by a segmentation algorithm. In practice segments include at 
least few tens of pixels. 

In order to compactly represent the motion vectors of the pixels in the 
5 segment it is desireable that their values are described by a function of few 
parameters. Such function is called motion vector field model. A known group 
of models are linear motion models, in which motion vectors are linear 
combinations of motion field basis functions. In such models the motion 
vectors of image segments are described by a general formula: 

10 ' =1 (3) 

Ay(x 9 y) = ^cJiix.y) 

where parameters q are called motion coefficients and are transmitted to the 
decoder. Functions fj(x,y) are called motion field basis functions which are 
fixed and known both to encoder and decoder. 

The problem when using the linear motion model having the above 

15 described formula is how to minimize the number of motion coefficients q 
which are send to the decoder, preserving at the same time as low measure 
of prediction error E n (x,y) as possible. This process is performed in the 
encoder by the motion field coding block 3, see fig. 1 . It is performed after 
computationally very complex motion field estimation which is accomplished 

20 by the block 2. It is therefore crucial that motion field coding is computationally 
simple so that it does not impose additional burden on the encoder. 

The total number of motion coefficients which needs to be sent to the 
decoder depends both on number of segments in the image and number of 
motion coefficients per segment. Therefore, there is at least two ways to 

25 reduce the total number of motion coefficients. 

The first way is to reduce the number of segments by combining 
(merging) together those segments which can be predicted with a common 
motion vector field without causing a large increase of prediction error. The 
number of segments in the frame can be reduced because very often the 
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adjacent, i.e. neighbouring, segments can be predicted well with the same set 
of motion coefficients. Process of combining such segments is called motion 
assisted merging. 

The second way is to select for each segment a motion model which 
5 allows achieving satisfactorily low prediction error with as few coefficients as 
possible. Since the amount and the complexity of the motion varies between 
frames and between segments it is not efficient to use always all A/+ M motion 
coefficients per segment. It is necessary to find out for every segment what is 
the minimum number of motion coefficients which allows to achieve 
10 satisfactorily low prediction error. The process of such adaptive selection of 
coefficients is called motion coefficient removal. 

Figure 3 shows a frame divided into segments. The prior art techniques 
for motion coefficient coding include several techniques for motion assisted 
merging. After motion vectors of all the segments have been estimated, the 
15 motion assisted merging is performed. It is done by considering every pair of 
adjacent segments S, and Sj with their respective motion coefficients c, andc ; . 

Area of combined segments S, and S. is denoted S. . If the area S u can be 

predicted with one set of motion coefficients c y without causing excessive 

increase of prediction error over the error resulting from separate predictions 
20 of 5, and 5,, then % and Sj are merged. The methods for motion assisted 

merging differ essentially in the way of finding a single set of motion 
coefficients c jy which allow a good prediction of segments combined together. 

One method is known as merging by exhaustive motion estimation. This 
method estimates "from scratch" a new set of motion parameters c f> for every 

25 pair of adjacent segments S. and . If the prediction error for is not 

excessively increased then the segments 5, and 5. are merged. Although this 

method can very well select the segments which can be merged it is not 
feasible for implementation because it would increase the complexity of the 
encoder typically by several orders of magnitude. 



WO 97/16025 



PCT/FI95/00581 



5 

Another method is known as merging by motion field extension. This 
method tests whether area of S u can be predicted using either motion 

parameters c, or c ; without an excessive increase of the prediction error. This 

method is characterised by very low computational complexity because it 
5 does not require any new motion estimation. However, it fails to merge 
segments very often because motion compensation with coefficients 
calculated for one segment very rarely predicts well also the adjacent 
segments. 

Still another method is known as merging by motion field fitting. In this 

10 method the motion coefficients c 0 are calculated by the method of 
approximation. This is done by evaluating few motion vectors in each of the 
segments. Some motion vectors in segments Sj and Sj are depicted in figure 
3. The motion field for the segment Sy is done by fitting a common motion 
vector field through these vectors using some known fitting method. The 

15 disadvantage of the method is that motion field obtained by fitting is not 
precise enough and often leads to unacceptable increase of prediction error. 

Methods for performing motion estimation with different models and 
selecting the most suitable one are proposed in the documents H. Nicolas and 
C. Labit, "Region-based motion estimation using deterministic relaxation 

20 schemes for image sequence coding," Proc. 1994 International Conference 
on Acoustics, Speech and Signal Processing, pp. III265-268 and P. Cicconi 
and H. Nicolas, "Efficient region-based motion estimation and symmetry 
oriented segmentation for image sequence coding," IEEE Tran. on Circtuits 
and Systems for Video Technology, Vol. 4, No. 3, June 1994, pp. 357-364. 

25 The methods try to adapt the motion model depending on complexity of the 
motion by performing motion estimation with different models and selecting 
the most suitable one. The main disadvantage of these methods is their high 
computational complexity and low amount of different motion field models 
which can be tested in practice. 

30 None of afore described methods alone solves the problem how to 

minimize the number of motion coefficients q which are send to the decoder, 
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preserving at the same time as low measure of prediction error E n (x,y) as 
possible. 



SUMMARY OF THE INVENTION 
5 An objective of the present invention is to create a motion field coder 

which reduces the amount of motion field vector information produced by 
some known motion estimator by large factor without causing large increase 
of prediction error. The complexity of the motion field coder should be low for 
allowing practical implementation on available signal processors or general 
1 0 purpose microprocessors. 

In accordance with the invention, the motion field coder includes three 
main blocks. 

The first main block is called a QR motion analyzer. Its task is to find a 
new representation of the inputted motion field produced by the motion field 

15 estimator. This new representation is applied to the second main block. Ope- 
rations in this first main block include a plurality of steps comprising matrix 
operations: In the first step the prediction frame is linearized using some 
known approximation method so that the prediction frame becomes linear with 
respect to motion vectors. In the second step a matrix Ej and a matrix y are 

20 constructed for minimisation of the square prediction error. In the third step 
the well known QR factorization algorithm is used to decompose the matrix Ej 
into a product of two matrices Qj and Rj. In addition, an auxiliary vector Zj is 
calculated from the factor matrix Qj and the matrix Part of the matrix Rj and 
the auxiliary vector Zj are applied to the second main block. 

25 The secod main block called a segment merging block performs merging 

operation for pairs of adjacent segments by finding whether their combined 
area can be predicted using a common motion field. In the matrix operations a 
matrix equation is firstly formed, thereafter the factor matrices are processed 
by using known matrix computation methods. The result is a matrix equation, 

30 where one matrix includes terms on the basis of which it is easy to calculate 
the sqare prediction error in the area of the merged segments. If the change 
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of the square prediction error is acceptable according to a chosen criterion the 
the segments are merged. 

After all pairs of segments are considered the output of the segment 
merging block is : 
5 L a new division of the image with reduced number of segments, 

ii. for each new segment the block outputs matrix R\> , vector z\> , 

iii. merging information which is sent to the decoder and helps the decoder to 
identify segments which were merged. 

The third main block is called a coefficient removal block. The block 

10 receives as the input new division of the current frame into segments and for 
every segment it receives matrices z\, andc* produced by the segment 
merging block. Motion vectors of every segment are represented by a number 
of motion coefficients. The motion coefficient removal block determines for 
each of the segments if it is possible to simplify the motion field model, wit- 

15 hout excessive increase of prediction error. Some basis functions are remo- 
ved from the motion model, wherein less coefficients are required to describe 
such a simplified motion field model. 

The operations in this third main block are matrix operations, in which 
the matrix equation is firstly modified by removing one column and row of the 

20 factor matrices, then the matrix equation is triangularized. Removal one co- 
lumn and row correspods to removal one basis function from the motion mo- 
del. The change of the square prediction error for the segment caused by re- 
moval of one basis function is equal to the square of one term in equation. 

If the change of the prediction error is acceptable according to a chosen 

25 criterion than one coefficient is removed from the coefficient set. More coeffi- 
cients for the segment can be reduced further repeating these matrix opera- 
tions. After sufficient amount of coefficients has been removed final motion 
coefficients for the segment are calculated by solving the resulting linear 
equation. The equation can be solved using one of well known algorithms, 

30 e.g., backsubstitution. 

This third main block outputs for every processed segment selection in- 
formation which tells which basis functions were removed from motion field 
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model. Additionally it outputs new motion coefficients corresponding to the 
remaining basis functions. Both selection information and motion coefficients 
are transmitted to the decoder. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram of a known encoder; 
Fig. 2 is a schematic diagram of a known decoder; 
Fig. 3 depicts adjacent segments for merging; 
Fig. 4 illustrates merging by motion field approximation; 
1 0 Fig. 5 is the motion field coder according to the invention; 

Fig. 6 is a schematic diagram of QR motion analyser. 

DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENT 

Fig. 5 illustrates the motion field encoder according to the invention. It 

15 corresponds to the block 3 in fig. 1 but it has as inputs also the reference fra- 
me and the current frame. The third input to this block is the motion vector 
field [Ax(-),Ay(0] produced by motion field estimation block 2, fig. 1. 

If we imagine, that the output of the video encoder is the compressed 
frame divided into segments and each of the segments is accompanied by 

20 motion coefficients, then for a segment 5. which consists of P pixels with 
coordinates * = 1,2,...,P, the task of the motion field coder is to find 

the motion coefficients c = (c p c 2 ,...,c„ +Af ) of a compressed motion vector field 

[Ar(-),AyO] where the motion vectors are described by a linear motion model, 
the field being of the form: 

y) = X Cifi (*■ y) (4a) 
25 1=1 

h(x,y)= 2 ? i/i(*.30 (4b) 

such that it minimizes square prediction error: 

p 

X (/« (*i . 3>< r ) - (*i + Ax(* 4 - , y 9 ), jv, + AjK*. . y, ))) 2 • (5) 
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15 



To fulfill said task, the motion field encoder consists of three main buil- 
ding blocks which are the QR motion analyser block, the segment merging 
block and the motion coefficient removal block. The segment merging block 
and the motion coefficient removal block reduce the amount of motion infor- 
mation which results in increase of the square prediction error. 

The objective of the QR motion analyzer is to find a new representation 
of the motion field. This new representation is used later in the other two 
blocks for a fast and flexible finding of motion coefficients for merged 
segments and for coefficient removal. 

The operation of QR Motion Analyzer consists of following steps: 

Step 1 is linearization of error. In this step the prediction frame /„_,(•) in 
equation (7) is approximated using some known approximation method so 
that it becomes linear with respect to [2^(0, Ay(-)]- Then elements under sum 

in the formula (5) become linear combinations of coefficients c, 

p 



(6) 



Step 2 is construction of matrices. It is based on the fact that minimisati- 
on of the formula (6) is fully equivalent to minimisation of the matrix expressi- 



E = 



"2,1 



-1,2 



-2,2 



are: 












y, = 






yp. 



(7) 



f>,\ *P,2 *-P,N+M_ 

20 Step 3 is QR Factorization. The well known QR factorization algorithm is 

described in the document G. H. Golub and C. van Loan, " Matrix computati- 
on" 2'nd edition, The Johns Hopkins University Press, 1989. This algorithm is 
used to decompose E, into a product of two matrices 

E.^Q.R, (8) 
25 In this step is also calculated an auxiliary vector z, where 

z, = Q r .y, (9) 
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In step 4 the output of the QR motion analyzer block is calulated. The 
output comprises of a matrix R/ consisting of N+M first rows of matrix R. and 
a vector z\ consisting of first N + M elements of z,.. 

In the segment merging block merging operation is performed for pairs 
5 of adjacent segments 5. andS y , see fig. 4, by finding whether their combined 
area S tj can be predicted using a common motion field described by motion 
coefficients c ij . Merging operation consists of the following steps: 

Step 1 comprises matrix calculation. This invention utilizes a previously 
unknown property that motion coefficients c kj can be found by solving the 
1 0 system of linear equations: 



R'i 













(10) 



where R l iy z\ and R 1 ;, z 1 , are already produced by the QR analyzer block for 
segments S, and Sj respectively. 

Step 2 comprises triangularization of the matrices obtained in the step 1 . 
15 Matrices R\, R 1 , are upper triangular and the system in equation (10) has 
according to the teaching of the afore mentioned documet the form: 



X 


X 


X • 


• X 




Z'l 




X 


X • 


• X 




Z'2 






X • 


♦ X 






Z'3 
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X 


X • 


X 

• X 


c 2 




Z' N+M 

zU 




X 


X • 
X • 


• X 
X 

X 


Sn+m _ 




Z j 2 
Z J 3 

_Z. } N+M 



(11) 



where symbol x denotes a nonzero element. According to the teaching of the 
afore mentioned documet, the system is triangularized by applying a series of 
20 multiplications of rows by scalars followed by additions of the rows, i.e., it is 
converted to the form: 
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1,1 


1,2 
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0 


^2,2 


r 2,3 


... 
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r 3.3 
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r N+M,N+M 
















C 3 
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0 






4l 
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0 






12 
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0 


0 


_ C N+M _ 




-73 


0 


0 


0 


0 


0 






0 


0 


0 


0 


0 




J1n*m 



(12) 



In the step 3 merging error is evaluated. The change of the square pre- 
diction error AE U in the area S 0 caused by merging of segments S i and 5 y is 

calculated according to the teaching of the afore mentioned documet, wherein 

5 AE 0 = (13) 

Finally, in the step 4 the segments are merged if the change of square 
prediction error in the formula (13) is acceptable according to a chosen criteri- 
on. For the resulting new segment S 0 matrix R\ y and vector z\ } are built by 
taking first N+M rows of the system in equation (12), i.e., are given by formula: 



1,2 


r i.3 ' 


r \,N+M 






r 2,2 


r 2,3 " 














, Z ij — 








r N+M,N+M m 







After all pairs of segments of the frame are considered the output of the 
segment merging block is obtained. The output comprises of three kind of in- 
formation. Firstly, it gives a new division of the image with reduced number of 
segments. Secondly, it gives for each new segment the block outputs matrix 
15 R\> , vector z l u Thirdly, it gives merging information which is sent to the deco- 
der and helps the decoder to identify segments which were merged. 
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The motion coefficients c 0 =(c lf c 2 ,...c w+w ) for the segment S {j could be 
now calculated by solving the system of equations R^c^ =z\j but their calcu- 
lation is not necessary if the next block, the coefficient removal block is used. 

We consider now operation of the coefficient removal block. The block 
5 receives as the input new division of the current frame into segments and for 
every segment S k it receives matrices R 1 *, z 1 *, andc^ produced previously by 

the segment merging block. Motion vectors of every segment is represented 
by N + M motion coefficients. 

The motion coefficient removal block determines for a given segment S k 

10 if it is possible to simplify the motion field model, without excessively increase 
of prediction error. Simplified motion field model is obtained when some basis 
functions are removed from the model in equations (3) described in the 
background art of this application. Less coefficients are required to describe 
such a simplified motion field model. 

15 The following procedure is performed for segments in order to find out 

whether i'th basis function (and i'th coefficient) can be removed from the moti- 
on field model: 

Step 1 includes matrix modification, where system of linear equations 

R^c, =z l k (15) 

20 is modified by removing i'th column from R 1 * and i'th element are from c k . 

Step 2 includes matrix triangularization, where the system in equation 
(15) is triangularized in the known manner by applying a series of multiplica- 
tions of rows by scalars followed by additions of the rows, i.e., it is converted 
to the form: 



25 



1.1 


1.2 


1.3 • 


r \,N+M-\ 














c, 
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1.2 


1,3 • 


r 2,N+M-\ 
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C 2 
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C 3 
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r N+M-l,N+M-\ 






^N+M-i 
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_ C N+M-\ _ 




4. . 



(16) 
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Step 3 includes error evaluation. The change of the square prediction er- 
ror for the segment caused by removal of i'th coefficient is simply equal to the 
term q 2 t in equation (16) . 

Step 4 includes removal of coefficients. If the change of the prediction 
5 error is acceptable according to a chosen criterion than the coefficient c, is 
removed from the coefficient set. The new number of coefficients is now N+M- 
1. Matrix R\ and vector z 1 * are modified to form: 





r U 1,2 


r u * 


I.W+Af-l 




z, 




r 2,2 


r 2,3 


r 2,N+M-l 




*2 



























Number of coefficients for the segment can be reduced further by using matri- 
10 ces (17) in the equation (15) and repeating the steps 1-4. 

Step 5 includes coefficient calculation. After sufficient amount of coeffi- 
cients has been removed, this step is started. In this step final motion coeffi- 
cients for a segment S k are calculated by solving the system of linear equa- 
tions: 

15 R l k c k =z l k (18) 

where matrix R\ and vector z 1 * are the result of the previous steps 1-4. 
The equation can be solved using one of well known algorithms, e.g., back- 
substitution. 

The motion coefficient removal block outputs for every processed 
20 segment selection information which tells to the decoder which basis functions 
were removed from motion field model. Additionally it outputs new motion 
coefficients corresponding to the remaining basis functions. Both selection 
information and motion coefficients are transmitted to the decoder. 

As a result of all of the steps in all of the blocks, the motion field coder 
25 according to the invention produces merging information telling to the decoder 
which segments are merged, selection information telling to the decoder 
which basis functions are removed and motion coefficient information. 
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The main advantage of the present invention over prior art solutions is its 
ability to reduce the amount of motion information by large factor without cau- 
sing large increase of prediction error. Additionally the complexity of the ove- 
rall system is low which allows practical implementation on available signal 
5 processors or general purpose microprocessors. 

The segment merging block has a unique ability of finding motion vec- 
tors of combined segments from given motion vectors estimated for separate 
segments. It can be proven that the motion vectors it produces are in fact op- 
timal in terms of keeping minimal square error for the combined segment. This 

10 explains the ability of this block to dramatically reduce the number of 
segments with only very modest increase of square prediction error. 

The motion coefficient removal block is a very powerful tool for instanta- 
neous adaptation of the motion model to the actual amount and type of moti- 
on in the video scene. This block can easily test the result of prediction (value 

15 of square prediction error for a segment) with a very large number of models, 
e.g., with all possible combinations of motion field basis functions. No other 
known method has this amount of flexibility. A strong advantage of this sche- 
me is that it does not need to repeat process of motion estimation hence is 
computationally simple. 

20 By using motion estimation followed by QR motion analyzer the motion 

field coder can find new motion coefficients for any desired combination of 
image segments or any desired model of the motion field in the segment by 
solving very simple systems of linear equations. 

25 DESCRIPTION OF THE PREFERRED EMBODIMENT 

In the preferred implementation quadratic polynomial motion vector field 
model with 12 coefficients is used: 

Ax(x, y) = c, + c 2 x + c 3 y + c 4 xy + c 5 x 2 + c 6 y 2 

a>>(x, y) = c 7 + c % x + c *y + c io*y + c i i * 2 + c ny 2 

This model in practice can handle very well even very complex motion in 
30 video sequences and yields good prediction results. 
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In the QR motion analyzer block, linearization in the step 1 is done by 
using Taylor expansion of /„_,(•) at every pixel (x^yj where i = l,2,... f P 
around points: 

= + Ax(x |i y i ) 
y/ = y f + Ay(jc p y 4 .) 

Using property that = ^(-a) 2 , the prediction error is then 
Auxiliary values g;(x,y) are calculated using formula: 



_ J fM^yG^y,) when ; = 1,2 N 

g ' (Wi) [//W/)G y (W.) when j = N + 1,N + 2,...,N + M 

where function fj(Xj,yj) is basis function as defined in the equations (4a) and 
10 (4b). 

Matrix E and vector y in equation (9) are build using formulas: 



E = 



gj(x 2 ,y 2 ) g 2 {x 2 ,y 2 ) 



y = 



g\(x p ,y P ) g 2 (x P ,y P ) ••• g N + M (x P ,y P )_ 

h (■*• . y, ) - y0 + G * (*•'. y,')Ax(*, , y, ) + G y (x,', y,')Ay(x, , y, ) 

/„ (^2 . y 2 ) - 7„-i . y0 + G x . y; )A*(* 2 . y 2 ) + (*; . >J )Ay(* 2 , y 2 ) 



/. (*#• • yp ) - Li ,y F ) + G x (x' py y' P )Ax(x P ,y P ) + G y (x' P , y' P )Ay(x P , y p )_ 
G x (x,y) and G y (x,y) are values of the horizontal and vertical gradient of the 
15 reference frame 7 n _^x,y) calculated using following formula: 

G x (x, y) = 7„_, ( x + 1, y) - 7„_, (x - 1. y), 
G y (x, y) = /„., (x, y + 1) - 7_, (*. y - D- 
Figure 6 shows a schematic diagram of QR Motion Analyzer. Row selec- 
tion block selects only N+M first rows of the input matrix. 
In the segment merging block following strategy for segment merging is used: 
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a. a threshold T is selected which corresponds to allowed increase of square 
prediction error in whole frame. 

b. for all pairs of adjacent segments AE ij is calculated using equation (13). 

c. pair of segments with smallest AE U is merged. 

5 d. points b-c are repeated until sum of A£. corresponding to all merged pairs 
of segments is greater then T. 

For triangularization of system in equation (11) a sequence of Givens 
rotations is used. 

In the motion coefficient removal block the following strategy for coeffi- 
10 cient removal is used: 

a. a threshold T is selected which corresponds to allowed increase of square 
prediction error in whole frame. 

b. for all segments and all basis functions q* is calculated using equation 
(16). 

15 c. basis function of a segment with smallest is removed. 

d. points b-c are repeated until sum of all q? 's corresponding to all removed 
basis functions in different segments is greater then T. 

The system in formula (16) is triangularized using a sequence of Givens 
rotations. 

20 The final motion coefficients of segments are calculated by solving 

equation (18) using backsubstitution algorithm. 

The pixel values of /„_,(*, >>), G x (x y y) and G y (x 9 y) are defined only for in- 
teger coordinates x and y . In many instances when x or y are noninteger the 
pixel value is calculated using bilinear interpolation of closest pixels with inte- 
25 ger coordinates. 

The system can be implemented in a variety of ways without departing 
from the spirit and scope of the invention. For instance, different linear motion 
models can be used in equations (3). Different methods can be used to linea- 
rize the term in the formula (5). Further, different criteria may be used to deci- 
30 ded whether to merge or not to merge two segments. The strategy for deci- 
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ding whether a given basis function should be removed from the model may 
vary. Triangularization of matrices in equations (10) and (15) can be perfor- 
med using various algorithms and calculation of final coefficients by solving 
equation (18) can be done using a number of known algorithms for solving 
5 systems of linear equations. Finally, different interpolation methods can be 
used to values of y). G x {x,y) and G y (x y y) in noninteger coordinates. 
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We claim: 

1 . A video codec comprising: 

a motion field coder responsive to the current frame, a reference 
frame and a motion vector field for producing a coded motion vector field and 
5 for minimizing the number of the motion coefficients, 
the motion field coder comprising: 

a first main block including linearization means for linearizing a squa- 
red error function between a segment of the current frame and a segment of 
the prediction frame, and matrix processing means for forming a first matrix 
10 R/ and a first vector z\, said matrix and vector being a new representation of 
the motion vector field of the segment; 

a second main block connected to the first main block, the second 
main block including merging means for merging pairs of adjacent segments if 
the combined area can be predicted using a common motion field, wherein, 
15 for each merged segment, the block outputs a first merged matrix R^ and a 
second merged vector zS, , 

a third main block connected to the second main block, the third 
main block including coefficient removal means for removing motion coeffi- 
cients of the segment untill the change of the squared prediction error is not 
20 acceptable, wherein the third main block outputs reduced number of motion 
coefficients of each segment. 

wherein in the outputted coded motion vector field each motion 
vector is based on a linear motion model comprising basis functions and mo- 
tion coefficients 

25 2. A video codec as in claim 1 , wherein linearization means trans- 

form the square prediction error to a form according to the formula: 

p 

where c i is motion coefficient and N+M is the number of motion coefficients. 

3. A video codec as in claim 2, wherein the matrix processing means 
30 include means for constructing following matrices: 
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10 





e i,2 • 


e i.N+M 






e 2.\ 


e 2,2 


e 2,N+M 


y, = 




fp.x 


e P,2 


e P,N+M_ 




yp. 



E ; = 



4. A video codec as in claim 3, wherein the matrix processing means 
further include QR factorization means for decomposing in the as such known 
manner the matrix Ej into a product of a matrix Q» and a matrix Rj and for for- 

5 ming an auxiliary vector Zj of the form z i = Q T i y i and for outputting a first 
matrix R, 1 consisting N+M first rows of matrix R, and the first vector z\ con- 
sists of first N+M elements of z.. 

5. A video codec as in claim 1 , wherein the merging means includes 
means for triangularizating the linear equation 

R\ 



R 1 ; 



•-[.'■':] 



where R\, z 1 / and R 1 ,, z ] j are produced by the first main block for segments 
S t and Sj respectively, to the form: 
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6. A video codec as in claim 5, wherein the merging means further 
15 includes means for calculating the change of the square prediction error 
AE ij in the area S 0 caused by merging of segments 5. and Sj in accordance 

with the following formula: 
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wherein the segments are merged if the change of the square prediction error 
is acceptable according to a given criterion. 

7. A video codec as in claim 5, wherein the merging means further 
includes means for building for the resulting merged segment S v a matrix R\j 

5 and a vector z\> by taking first N+M rows of the triangularizated matrix equati- 
on, wherein the matrix R 1 */ and vector are described by the following formula: 



'1.2 



'3,3 



'l,N+M 
r 2,N+M 



'3,N+M 



'N+M, N+M 



*3 



L Z N+M J 



said matrix and vector being outputted from the second main block. 

8. A video codec as in claim 1, wherein the merging information is 
1 0 send to the decoder. 

9. A video codec as in claim 1 , wherein the coefficient removal me- 
ans includes means for reducing the matrix equation R 1 *^ = z\ by removing 
i'th column from a matrix R 1 * and i'th element from the vector c, , 

10. A video codec as in claim 9, wherein the coefficient removal me- 
15 ans further includes means for triangularizing the reduced matrix equation to 

the form: 



r l.I 


r i,2 


r L3 


r \,N+M-l 






z. 


0 


r 2,2 


r 2,3 


r 2,N+M-\ 
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r N+M-\.N+M-l 
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wherein the change of the square prediction error caused by removal of the 
i'th column and Pth element is equal to <? 2 , . 

1 1 . A video codec as in claim 10, wherein the coefficient removal means 
further includes means for comparing the change of the prediction error to a 
chosen criterion. 
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12. A video codec as in claim 1 1 , wherein the coefficient removal means 
further includes means for removing the motion coefficient c t from the coeffi- 
cient matrix if the change is acceptable, and means for forming a modified 
matrix R\ and a modified vector z\ by removing the bottom row from the 

5 triangularized matrix, said modified matrix and modified vector being applied 
to the coefficient removal means. 

13. A video codec as in claim 1 1 , wherein the coefficient removal means 
further includes means for calculating of final motion coefficients for the 
segment by solving the system of linear equations R'ac* -z\ , if the further 

10 change is not acceptable. 
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